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Robustness  of  Path- Vector  Protocols  without 
Independent  Route  Ranking 

Aaron  D.  laggard*  Vijay  Ramachandran^ 


Abstract 

Recent  work  has  presented  theoretical  frameworks  that  rigorously  model 
the  behavior  of  path-vector  protocols  [6,7,  13],  which  are  primarily  used 
for  inter-domain  routing  on  the  Internet.  We  expand  the  scope  of  these  to 
include  protocols  with  route-selection  procedures  that  cannot  be  captured 
by  a  per-node  linear  order  on  paths;  in  particular,  our  generalized  model 
captures  the  use  of  commonly  deployed  route  attributes  such  as  MED,  which 
is  used  to  fine-tune  routing  between  networks  that  share  more  than  one  inter¬ 
connection.  Using  the  model,  we  give  the  best-known  sufficient  condition 
guaranteeing  robust  convergence  of  path-vector  protocols  in  the  generalized 
case  and  discuss  its  applications  to  protocol  design. 


1  Introduction 

Most  routes  on  the  Internet  transit  several  independently  administered  network  do¬ 
mains,  called  autonomous  systems  (ASes).  Although  routing  within  an  AS  is  well 
understood,  routing  between  ASes  is  difficult  because  of  the  diverse  networks  and 
larger  distances  involved.  Inter-domain  routing  on  the  Internet  is  accomplished  to¬ 
day  using  the  Border  Gateway  Protocol  (BGP)  [12],  a  path- vector  protocol.  Routes 
are  established  hop-by-hop  through  the  network  as  information  about  destinations 
is  shared  between  routers;  at  each  step,  this  depends  on  routers’  locally  configured 
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routing  policies,  which  can  be  quite  expressive.  Therefore,  convergence  to  a  sta¬ 
ble  set  of  consistent  routes  throughout  the  network  is  dependent  on  a  composition 
of  decisions  involving  many  different,  autonomously  provided  inputs.  Previous 
work  [14]  has  demonstrated  that  the  interaction  of  these  local  policies  can  produce 
global  anomalies  in  BGP,  e.g.,  nondeterministic  routing  and  protocol  divergence. 
To  acheive  greater  network  stability,  a  better  understanding  of  the  interaction  of 
routing  policies  is  necessary;  furthermore,  this  must  be  done  in  a  rigorous  manner 
so  that  network  operators  can  rely  on  provable  guarantees  about  protocol  behavior, 
even  in  worst-case  scenarios. 

This  paper  continues  a  line  of  work  that  explores  the  theoretical  foundations 
of  inter-domain  routing  and  routing-policy  interaction.  Several  existing  models 
of  path-vector  protocols  [5-7, 13]  that  are  used  to  derive  general  sufficient  condi¬ 
tions  for  robust  convergence  ignore  the  complexities  of  sharing  inter-domain  routes 
within  an  AS;  in  particular,  the  model  of  the  Internet  assumes  that  every  AS  can  be 
represented  by  one  node  in  a  graph  with  a  single  routing  policy  and  a  single  link 
to  each  neighboring  node.  (In  reality,  an  AS  is  often  made  up  of  several  routers 
that  maintain  BGP  sessions  to  share  inter-domain  routes;  these  sessions  often  con¬ 
nect  links  to  different  neighboring  ASes  and  provide  multiple  inter-connections 
between  the  same  ASes.)  On  the  other  hand,  work  that  included  these  complexi¬ 
ties  [1,8]  has  been  unable  to  prove  general  guidelines  for  convergence  in  the  same 
way  that  these  models  have.  Our  paper  bridges  this  gap  by  presenting  a  generalized 
model  to  capture  the  static  semantics  of  policy  interactions  for  both  inter-domain 
and  intra-domain  BGP  sessions.  This  model  is  used  to  derive  a  sufficient  condition 
analagous  to  those  for  the  simplified  case. 

In  the  remainder  of  this  section,  we  review  existing  theoretical  frameworks  that 
inspired  the  methodology  of  this  work  and  previous  attempts  to  analyze  anomalies 
related  to  multiple  AS  inter-connections,  discussing  a  specific  example  with  BGP. 
In  Section  2,  we  introduce  our  model,  defining  relevant  terms  (both  old  and  new). 
We  then  derive  the  generalized  convergence  constraint  in  Section  3  and  discuss 
various  applications  in  Section  4. 

1.1  Related  Work  on  Modeling  BGP 

Path-vector  protocols  establish  connectivity  by  sharing  putative  routes  to  reachable 
destinations  on  a  hop-by-hop  basis  between  routers:  potential  routes  are  collected; 
a  best  route  is  chosen;  and  that  choice  is  advertised  to  neighboring  routers  that 
repeat  the  process.  Local  policies  influence  each  step  of  this  process,  because  the 
attributes  of  a  route — information  in  a  path’s  data  record — is  changed  on  import 
and  export  based  on  policy  configuration,  and  this  affects  the  choice  of  the  best 
routes  and  which  routes  are  shared.  Gao  and  Rexford  [5]  showed  that  constraints 
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on  policies  and  a  simple  assumption  about  the  business-relationship  structure  of  the 
Internet  can  guarantee  robustness,  i.e.,  predictable  convergence  to  a  routing  solu¬ 
tion,  even  in  the  presence  of  link  and  node  failures.  The  benefit  of  this  result  is  that 
the  constraints  are  local,  i.e.,  network  stability  could  be  achieved  with  little  global 
coordination  between  policies,  which  is  normally  impossible  given  the  autonomy 
of  ASes. 

Griffin,  Shepherd,  and  Wilfong  [7]  presented  the  Stable  Paths  Problem  (SPP) 
as  the  underlying  theoretical  problem  being  solved  by  BGP  SPP  captured  the  static 
semantics  of  the  interaction  of  routing  policies  on  paths  to  a  single  destination  as  a 
preference  ordering  at  each  node  on  putative  routes.  They  were  able  to  give  a  suffi¬ 
cient  global  condition  for  robustness,  but  showed  that  checking  individual  policies 
exactly  for  the  existence  of  a  stable  routing  solution  is  AP-hard.  The  combination 
of  these  results  with  the  Gao-Rexford  conditions  produced  an  elegant  analysis  of 
the  behavior  of  a  modified,  safe  version  of  BGP  fhat  assumed  the  required  Internet 
business  structure  but  allowed  back-up  routing  [4]. 

These  results  were  incorporated  into  two  theoretical  frameworks  [6, 13]  that 
model  the  behavior  and  design  of  path- vector  protocols  more  generally;  both  of 
these  give  concrete,  rigorous  analysis  of  convergence  conditions.  They  show  that 
an  underlying  consistency  between  the  preference  ordering  of  routes  at  nodes  and 
ordering  routes  by  path  length  represents  a  sufficient  condition  for  robust  conver¬ 
gence  equivalent  to  that  of  the  original  SPP  work;  they  also  explain  how  to  achieve 
or  enforce  that  ordering  in  various  ways. 

1.2  MED-Induced  Oscillations 

Unfortunately,  the  above  branch  of  work  only  applies  to  a  specific  type  of  path- 
vector  protocol — those  in  which  the  best-route  selection  procedure  can  be  mod¬ 
eled  by  mapping  paths  under  consideration  to  a  rank,  or  weight,  in  some  totally 
ordered  set  and  choosing  the  path  of  minimum  (or  maximum)  rank.  This  prop¬ 
erty  is  called  independent  route  ranking  (IRR)  because  the  rank  of  a  path  can  be 
determined  from  the  attributes  of  that  path’s  data  structure,  which,  in  turn,  can  be 
used  to  compare  it  to  any  other  path  for  best-route  selection.  However,  BGP’s  full 
route-selection  procedure  cannot  be  modeled  in  this  way.  In  particular,  use  of  the 
multi-exit  discriminator  (MED)  attribute,  which  is  common  when  two  ASes  share 
multiple  inter-connections  and  want  to  perform  cold-potato  routing,*  violates  IRR. 

'Normally,  ASes  use  hot-potato  routing,  in  which  traffic  destined  for  another  AS  is  routed  to  the 
nearest  egress  point — or  border  router — with  a  connection  to  the  next-hop  AS  on  the  path.  In  the 
case  of  cold-potato  routing,  if  an  AS  has  two  or  more  connections  to  the  next-hop  AS,  some  factor 
other  than  shortest  distance  is  used  to  choose  the  egress  point.  The  MED  attribute  is  a  common  way 
to  force  cold-potato  routing,  and  its  use  will  be  discussed  in  more  detail  later  in  this  paper. 
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MED-induced  oscillations  are  a  well-known  problem  of  BGP  [2, 3, 10],  and  it 
has  been  conjectured  that  the  violation  of  independent  route  ranking  is  the  major 
reason.  These  oscillations  are  especially  difficult  to  analyze  and  debug  on  a  real 
network  because  they  are  a  product  of  not  only  BGP  policy  settings — involving  at¬ 
tributes  set  in  separately  configured,  independent  ASes — ^but  also  internal  distance 
settings  within  an  AS  (determined  by  an  interior  gateway  routing  protocol,  or  IGP). 

There  has  been  some  theoretical  work  on  the  consequences  of  using  the  MED 
attribute,  but  the  results  have  been  incomplete.  Basu  et  al.  [1]  and  Musunuri  and 
Cobb  [11]  proved  that  including  in  advertisements  routes  not  chosen  as  best  pre¬ 
vents  MED-induced  oscillations,  but  this  change  to  BGP  would  increase  the  size  of 
routing  tables  and  the  number  or  size  of  update  messages.  Griffin  and  Wilfong  [8] 
enumerated  canonical  examples  of  MED-induced  oscillations  and  described  them 
using  a  narrow  extension  to  their  SPP  model,  but  did  not  propose  broad  configura¬ 
tion  suggestions  for  using  the  MED  attribute  nor  a  robustness  constraint  analogous 
to  that  given  for  the  original  SPP  model.  Other  suggestions  to  solve  the  MED- 
oscillation  problem  affect  the  use  of  route  reflectors  and  configuration  of  iBGP 
sessions  within  an  AS  [15]  or  require  changing  the  interpretation  of  attributes  [10]. 

This  paper  provides  a  complete,  rigorous  model  that  not  only  permits  analysis 
of  the  MED  attribute  but  also  includes  more  general  route-selection  procedures 
that  may  violate  the  independent-route-ranking  property.  We  fully  extend  the  SPP 
framework  to  cover  these  instances  of  the  inter-domain  routing  problem  and  derive 
a  constraint  for  policy  configuration  that  guarantees  robust  convergence;  as  it  is 
more  general,  it  applies  to  instances  of  the  limited,  original  SPP  model  as  well.  The 
generalized  SPP  can  be  included  in  the  existing  path-vector  design  frameworks  [6, 
13]  to  broaden  their  application  and  give  a  much  cleaner  measure  for  protocol 
expressiveness. 

2  A  Generalized  Framework  for  Inter-Domain  Routing 

We  begin  this  section  by  reviewing  the  dynamics  of  inter-domain  routing  protocols. 
We  then  define  route-selection  functions  and  independent  route  ranking  (IRR),  ex¬ 
plaining  the  difference  between  our  more  general  definitions  and  the  more  spe¬ 
cific  definitions  used  in  previous  theoretical  work.  We  then  present  the  General¬ 
ized  Stable  Paths  Problem  (GSPP)  and  the  Generalized  Path- Vector  Policy  System 
(GPVPS)  as  the  underlying  theoretical  problem  being  solved  by  routing  protocols 
and  the  framework  for  routing-protocol  design,  respectively,  both  of  which  incor¬ 
porate  the  generalized  version  of  route  selection.  In  doing  so,  we  provide  an  exam¬ 
ple  GSPP  demonstrating  a  MED-induced  oscillation. 
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2.1  Overview  of  Inter-Domain  Routing 

Internet  traffic  is  forwarded  from  source  to  destination  by  routers  along  paths  that 
traverse  inter-domain  and  intra-domain  links.  Routers  perform  a  basic  forwarding 
operation,  in  which  the  destination  IP  address  of  a  packet  of  traffic  is  mafched  to 
an  entry  in  a  forwarding  table,  and  the  packet  is  sent  to  the  corresponding  next 
hop — or  neighboring  router — listed  in  the  entry.  The  job  of  routing  protocols  is  to 
fill  this  forwarding  table  to  form  consistent,  loopless  paths  for  traffic  to  follow. 

Intra-domain  routing  is  well  understood  and  is  often  based  on  simultaneous 
best-path  calculations  using  some  Interior  Gateway  Protocol  (IGP) — at  the  intra¬ 
domain  level,  “best”  is  often  defined  as  shortest.  Inter-domain  routing,  however,  is 
more  complicated  because  the  autonomy  of  domains  and  the  scale  of  the  Internet 
prevents  both  information  about  network  topology  to  be  distributed  for  such  cal¬ 
culations  and  coordination  or  consistency  among  definitions  of  “best.”  Therefore, 
routes  are  computed  on  a  hop-by-hop  basis  and  decisions  are  influenced  by  local 
policy  configurations. 

Knowledge  about  destinations  is  learned  through  advertisements  from  neigh¬ 
boring  routers;  once  a  path  to  another  AS  is  established,  an  AS  will  share  that 
reachability  information  with  its  neighbors  so  that  they  gain  knowledge  of  the  des¬ 
tination  as  well.  Assuming  that  destinations  are  first  originated  by  the  protocol¬ 
speaking  router  responsible  for  that  destination,  paths  are  thus  established  by  re¬ 
peating  the  following  three-step  process: 

1 .  Information  about  established  routes  through  neighboring  routers  is  collected, 
called  importing  routes.  The  route  data  stored  in  the  local  routing  table  de¬ 
pends  on  the  route  information  in  the  update  message  and  the  import  policy, 
the  policy  may  filter  routes  entirely,  i.e.,  remove  them  from  consideration. 

2.  For  each  destination,  the  protocol’s  best-route  selection  procedure  is  used  to 
choose  best  routes  from  the  local  routing  table.  Best  routes  are  then  used  to 
populate  the  forwarding  table  for  these  destinations. 

3.  Best  routes  are  advertised  to  neighboring  routers,  called  exporting.  Update- 
message  information  about  these  routes  is  influenced  by  export  policy,  which 
may  also  filter  routes. 

The  routers  with  inter-AS  connections  exchanging  this  information  are  border 
routers',  however,  an  AS  consists  of  non-border  routers  that  must  learn  how  to 
reach  external  destinations  as  well.  The  inter-domain  protocol  is  thus  also  used  to 
share  external  destinations  with  internal  routers.  As  a  result,  path-vector  protocols 
accomplish  two  inter-domain  routing  tasks: 
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1 .  establishing  connectivity  and  sharing  reachability  information  across  inter¬ 
domain  links;  and 

2.  distributing  knowledge  of  inter-domain  routes  to  non-border  routers. 

It  is  important  to  note  that  much  of  the  previous  theoretical  work  studying  the 
convergence  of  BGP  and  other  path- vector  protocols,  e.g.,  [5-7, 13],  modeled  the 
Internet  as  a  graph  in  which  each  vertex  represents  one  AS,  thereby  considering 
only  inter-AS  connections  and  ignoring  anomalous  behavior  related  to  task  (2). 
However,  such  anomalies  have  indeed  been  identihed  [2,  3,  10],  and  this  paper 
generalizes  the  previous  theoretical  work  to  address  these  anomalies. 

We  write  paths  in  the  direction  of  forwarding  traffic;  e.g.,  P  =  vqVi  •  •  •  is  a 
path  from  node  vq  to  destination  Vn-  Node  vi  is  the  next  hop  on  P.  At  the  inter¬ 
domain  level,  most  nodes  Vi  will  represent  ASes,  not  individual  routers.  However, 
because  of  task  (2),  it  will  be  important  to  write  a  portion  of  the  path  from  the 
source  router  to  the  border  router  such  that  nodes  represent  internal  routers;  e.g., 
we  may  write  P  =  AHC'(3)(6)(12)(7)  for  a  path  from  the  source  AS  starting 
at  router  A  through  internal  router  B  to  border  router  C,  then  onto  ASes  3,  6, 
and  12  before  reaching  the  destination  AS  7.  We  assume  that  each  of  transit  and 
destination  ASes  can  appropriately  route  traffic;  fhus  infer-domain  roufe-signaling 
messages  do  nof  confain  infra-domain  roufing  informafion  for  fhese  ofher  ASes. 
In  general,  when  a  roufer  is  esfablishing  forwarding  pafhs  fo  a  desfinafion,  we  can 
view  fhe  Infernef  graph  from  fhat  roufer’s  perspecfive  as  one  in  which  all  ofher  ASes 
are  represenfed  by  one  node,  neighboring  ASes  connecf  fo  fhe  border  roufers  of  fhis 
roufer’s  AS,  and  ofher  nodes  represenf  fhe  infra-domain  roufers  and  connecfions. 
(If  is  inferesfing  fo  nofe  fhaf  infer-domain  roufing  profocols  are  fhemselves  built  on 
top  of  IP;  therefore,  it  is  expected  that  intra-domain  IP  forwarding  can  take  place 
based  on  computations  of  the  IGP.) 

2.2  Route- Selection  Functions  and  Independent  Route  Ranking 

Step  2  in  the  above-described  three-step  process  of  choosing  best  routes  from  a 
routing  table  can  be  modeled  by  the  following  type  of  function. 

Definition  2.1.  A  route-selection  function  maps  a  set  of  paths  7?  to  a  set  S'  C  7? 
that  is  a  set  of  “best”  routes  at  node  v,  we  write  av{R)  =  S.  When  we  restrict  the 
selection  to  a  particular  destination,  we  will  write  a'^{R)  =  S*^  such  that  all  paths 
S'^  have  destination  d. 

In  most  cases,  including  BGP,  |cr'^(7?)|  <  1  for  a  set  of  paths  R  and  some  des¬ 
tination  d  (i.e.,  for  each  destination,  at  most  one  best  path  is  chosen).  Furthermore, 
we  assume  that  choosing  some  permitted  path  is  preferred  to  choosing  no  path. 
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although  some  paths  are  filtered  by  local  policy  such  that  they  are  never  considered 
as  part  of  the  selection  process.  Assuming  that  these  filtered  paths  are  not  stored 
in  the  routing  table  R,  then  for  all  R'^  C  R  to  a  particular  destination  d,  R^  ^  % 
implies  a'^{R'^)  /  0.  The  process  of  collecting  and  storing  routes,  including  what 
data  structures  are  used  for  this  purpose,  and  how  it  interacts  with  the  selection 
procedure  depend  on  the  protocol  implementation. 

Independent  route  ranking  (IRR)  means  that  the  preference  of  a  path  relative 
to  other  paths  depends  only  on  that  path  alone  (and  any  information  in  that  path’s 
routing-table  entry)  and  not  knowledge  of  other  paths. 

Definition  2.2.  A  selection  function  a  obeys  Independent  Route  Ranking  iff,  for 
all  sets  of  routes  Ri  and  i?2  and  destinations  d,  the  following  two  conditions  hold: 

1.  a'^{Ri)  =  S  implies  a'^{Ri  U  R2)  H  (i?i  \  S')  =  0;  and 

2.  a'^{Ri)  =  S  and  a'^{Ri  U  R2)  n  S  /  0  implies  U  R2)  5  S. 

We  call  violations  of  the  first  condition  type-1  IRR  violations  and  those  of  the 
second  condition  type-2  IRR  violations.  In  the  case  of  single-route  selection  func¬ 
tions,  the  above  definition  of  IRR  is  equivalent  to  the  following:  if  path  Pi  is  chosen 
over  all  paths  in  P  as  best,  then  additional  knowledge  of  a  route  P2  ^  P  does  not 
then  permit  another  route  P3  7^  Pi  in  P  to  be  chosen  as  best;  only  Pi  or  P2  may  be 
chosen  relative  to  P  U  {  P2  } .  (Condition  2  is  not  relevant  for  single- valued  selection 
functions.) 

Previous  theoretical  work  [6,7, 13]  on  path- vector  protocols  modeled  only  se¬ 
lection  functions  that  independently  assign  a  rank  to  each  route  and  choose  the  path 
of  minimal  (or  maximal)  rank.  Selection  functions  written  in  this  way  are  called 
linear  selection  functions',  at  each  node,  the  preference  order  on  unfiltered  (per¬ 
mitted)  paths  is  consistent  with  a  linear  order.  Because  the  protocol-convergence 
conditions  described  in  [6,7, 13]  depended  on  this  notion  of  rank,  they  do  not  apply 
to  the  more  general  setting  involving  arbitrary  selection  functions.  We  note  the 
relationship  between  linear  selection  functions  and  IRR  below. 

Definition  2.3.  A  selection  function  u  is  a  linear  selection  function  iff  there  exists 
a  map  uj  -.V  from  permitted  paths  P  to  a  totally  ordered  set  U  such  that 

VP  C  P,  a{R)  =  [P  I  VP'  G  P,  uj{P)  <  uj{P')]  . 

Proposition  2.4.  A  selection  function  has  no  IRR  violations  iff  it  can  be  written  as 
a  linear  selection  function. 

Proof.  First  assume  that  a  is  a  linear  selection  function;  then  there  exists  some 
ranking  function  uj  such  that  a{R)  =  {P  |  ijj{P)  <  w(P')  VP'  G  P}.  If  there 
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were  a  type-1  IRR  violation,  then  there  exist  Ri,  R2  such  that  a{Ri)  =  Si  and 
U  R2)  =  S2  such  that  S2  C  {Ri  \  Si).  But  then  for  all  P  G  S2,  it  must  be 
that  uj{P)  <  u;{P')  for  all  P'  G  {Ri  U  R2),  but  this  implies  that  uj{P)  <  u){P')  for 
all  P'  €  Ri,  thus  P  €  Si,  which  contradicts  S2  C  (i?i  \S'i).  If  there  were  a  type-2 
IRR  violation,  then  there  exist  Ri,  R2  such  that  a{Ri)  =  Si  and  a{RiUR2)  =  5*2 
such  that  ^2  n  51  /  5i  and  ^2  n  5i  /  0.  Let  P  G  5i  \  ^2;  then  w(P)  <  a;(P') 
for  all  P'  G  Pi  but  there  exists  P"  G  (Pi  U  P2)  such  that  oj{P)  >  oj{P").  Thus 
P"  G  P2,  but  by  transitivity,  if  oj{P)  <  ijj{P')  for  P'  G  Pi,  then  oj{P")  <  u!{P'); 
this  means  that  no  route  P'  G  Pi  could  be  chosen  by  c7(PiUP2),  which  contradicts 
52  n  5i  7^  0.  Therefore,  any  linear  selection  function  has  no  IRR  violations. 

Now  assume  that  we  have  an  IRR  selection  function  a.  For  all  P  C  P,  if 
cr(P)  =  5,  then  assign  path  ranks  such  that  w(P)  <  oj{P')  for  P  G  S,  P'  G 
R.  Suppose  this  ordering  of  path  ranks  is  not  consistent  with  a  linear  order;  then 
there  exist  two  permitted  paths  Pi,  P2  such  that  more  than  one  of  io{Pi)  <  uj{P2), 
u}{Pi)  >  uj{P2),  and  to’(Pi)  =  u;(P2)  are  true.  Suppose  that  (t({Pi,  P2})  =  {Pi} 
so  that  oj{Pi)  <  uj{P2),  but  w(P2)  <  w(Pi):  then  there  exists  R  C  V  such  that 
cr(P)  n  {Pi,P2}  =  {P2}  and  P  D  {Pi,P2}.  But  then  let  Pi  =  {Pi,P2}  and 
P2  =  P  \  {Pi,  P2};  then  cr  has  a  type-1  IRR  violation  with  Pi  and  P2,  which 
contradicts  our  IRR  assumption.  (By  symmetry,  there  is  also  a  contradiction  if 
a;(Pi)  >  w(P2)  but  a;(P2)  -ft  ^(Pi).)  Suppose  that  cr({Pi,  P2})  =  {Pi,P2} 
so  that  to’(Pi)  =  uj{P2)  but  uj{P2)  /  w(Pi):  then  there  exists  R  C  V  such  that 
cr(P)  {Pi,P2}  and  P  D  {Pi,P2}.  But  then  let  Pi  =  {Pi,P2}  and  P2  = 
P\  {  Pi ,  P2  } ;  then  a  has  a  type-2  IRR  violation  with  Pi  and  P2 ,  again  contradicting 
our  IRR  assumption.  Therefore,  any  IRR  selection  function  can  be  written  as  a 
linear  selection  function.  □ 

It  has  been  conjectured  that  IRR  violations  are  a  major  cause  of  protocol  os¬ 
cillations  [1,8].  Below  we  prove  that  given  one  IRR  violation  at  a  node,  we  can 
construct  a  simple  network  on  which  the  protocol  diverges,  but  in  which  the  other 
nodes’  selection  functions  obey  IRR  and  could  satisfy  previously  established  con¬ 
vergence  conditions. 

Theorem  2.5.  Suppose  av  is  an  IRR-violating  (nonlinear)  selection  function.  Then 
there  exists  an  oscillating  network  instance  containing  node  v  in  which  all  other 
nodes  have  IRR  (linear)  selection  functions. 

Proof.  Assume  single-valued  selection;  the  argument  below  generalizes.  If  a  vi¬ 
olates  IRR,  then  there  exists  a  set  Y  of  paths  to  d  containing  at  least  vPi,vP2 
such  that  cr'^(Y)  =  vPi  and  for  some  set  of  paths  Z  ib  such  that  Z  r\Y  =  0, 
cr'^{Z  U  y)  =  VP2.  Let  the  next  hops  on  vPi,vP2, ...  G  Y  vi,V2, . . .,  respec¬ 
tively;  assume  that  these  paths  are  fixed,  i.e.,\/  R  3  Pi,  a'^_^(R)  =  Pi  and  anal- 


ogously  for  the  other  a^..  Let  the  next  hops  on  vZi,vZ2, ...  G  Z  he  zi,  Z2,  ■  ■  ■, 
respectively.  For  each  Zi,  let  U  {Zi})  be  Zi  if  R  ^  ZivP2  and  ZivP2  if 

R  3  ZivP2',  but,  assume  that  Zi  are  fixed,  i.e.,  these  paths  are  always  broadcast. 

This  instance  diverges.  Assume  that  the  links  between  all  routers  use  first-in- 
first-out  (FIFO)  communication,  though  the  network  may  be  asynchronous.  Con¬ 
sider  a  snapshot  in  time  in  which  v  has  chosen  VP2  as  best;  this  only  happens  if 
it  also  leai'ns  of  all  paths  in  Z,  which  means  that  all  Zi  have  selected  Zi  as  their 
best  paths.  After  this,  v  will  broadcast  VP2  to  its  neighbors,  eventually  reaching 
the  Zi-  These  nodes  will  thus  switch  to  ZivP2,  withdrawing  Zi.  When  the  with¬ 
drawal  reaches  v,  it  will  switch  to  choosing  vPi  as  best,  withdrawing  VP2.  This 
withdrawal  will  eventually  reach  the  Zi  (after  the  first  broadcast  of  VP2)  causing 
these  nodes  to  switch  back  to  Zi ;  the  broadcast  of  these  choices  back  to  v  returns 
us  to  the  original  state  when  all  the  Zi  switch,  thus  producing  an  oscillation. 

If  we  examine  the  network  when  v  has  chosen  vPi  as  best,  there  are  two  possi¬ 
bilities:  (1)  All  Zi  have  chosen  Zi  as  best  but  the  broadcasts  of  these  choices  have 
not  yet  reached  v,  or  (2)  some  Zi  (possibly  all)  have  not  chosen  Zi  as  best.  In  case 
(I),  the  broadcasts  of  Zi  will  eventually  reach  v  causing  a  choice  of  VP2  as  best; 
this  leads  to  the  starting  state  above.  In  case  (2),  if  Zi  is  not  best  at  some  Zi,  ZivP2 
must  be  available  because  Zi  is  fixed;  thus,  v  must  have  chosen  VP2  at  some  pre¬ 
vious  time.  In  this  case,  choose  that  snapshot  of  time  as  a  starting  point,  and  it  is 
clear  that  the  above  oscillation  will  occur.  □ 

2.3  Generalized  Stable  Paths  Problem 

The  Stable  Paths  Problem  (SPP)  [7]  was  suggested  as  the  theoretical  problem  un¬ 
derlying  inter-domain  routing,  but  limits  nodes’  route-selection  functions  to  linear 
selection  functions.  We  now  present  the  generalized  version  first  discussed  in  [8] 
to  accommodate  modeling  attributes  in  BGP  that  are  inconsistent  with  independent 
route  ranking. 

Definition  2.6.  An  instance  of  the  Generalized  Stable  Paths  Problem  (GSPP)  is 
a  network  G  =  {V,E)  and  a  set  of  permitted  paths  "P  in  G  to  a  fixed  desfinafion 
node  vq  G  V.  (The  sef  V  of  permiffed  pafhs  can  be  parfifioned  info  sefs  Vy,v  £  V, 
which  are  fhe  permiffed  pafhs  af  node  v,  i.e.,  sfarfing  af  v  and  ending  af  uq.)  All 
nodes  v  ^  vq  have  a  roufe-selecfion  funcfion  :  2^”  —>■  V^.  A  path  assignment 
TT  :  V  ^  P  is  a  solufion  fo  GSPP  iff  7r(uo)  =  (vq)  and  for  every  u  /  no  G  P, 
7r(u)  =  (T^°  {{vP  G  P  I  P  =  7r{u)  and  {u,  u}  G  E}). 

Remark  2.7.  GSPP  is  AP-complefe.  This  is  because  GSPP  is  in  NP — given  a 
solufion,  if  is  easy  fo  check  whefher  if  is  sfable — and  because  SPP,  an  AP-complefe 
problem  [7] ,  frivially  reduces  fo  GSPP  by  wrifing  ifs  path  preferences  as  (linear) 
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selection  functions.  Also  note  that  this  version  of  the  problem  assumes  single¬ 
valued  selection  functions. 

Example  2.8.  Figure  1  shows  an  example  GSPP  given  in  [8, 10].  This  instance 
models  the  route-selection  procedure  of  BGP  running  on  a  network  in  which  the 
Multi-Exit  Discriminator  (MED)  attribute  is  used.  The  network  is  shown  from  the 
perspective  of  AS  3.  We  briefly  discuss  the  BGP  route-selection  procedure  and 
then  discuss  this  specific  GSPP. 

When  a  route  is  imported  (or  learned)  from  neighbors,  it  is  given  a  local- 
preference  value  that  is  entered  into  the  routing  table  to  indicate  how  “good”  the 
route  is;  the  MED  attribute,  on  the  other  hand,  is  set  by  the  exporting  (or  adver¬ 
tising)  AS  to  indicate  its  preference  among  multiple  inter-AS  connections.  The 
path-selection  procedure  for  BGP  is  as  follows: 

1 .  Routes  with  the  largest  local  preference  are  chosen  as  best. 

2.  In  the  case  of  a  tie,  routes  with  the  shortest  AS-path  length  are  chosen. 

3.  In  the  case  of  a  tie,  if  there  are  multiple  paths  to  the  same  AS,  choose  the 
path  with  the  lowest  MED  value.  MED  values  are  only  compared  among 
paths  to  the  same  AS. 

4.  If  there  remains  a  tie  because  there  are  paths  to  different  ASes,  choose  the 
path  with  the  shortest  IGP  distance  to  its  egress  point. 

Therefore,  the  importing  AS  has  ultimate  authority  by  setting  local-preference  val¬ 
ues,  but  these  are  often  set  to  the  same  value  for  all  routes  through  given  AS,  even 
across  different  inter-AS  links.  In  practice,  this  allows  a  neighboring  AS  to  influ¬ 
ence  the  decision  between  the  inter-AS  links  using  the  MED  attribute. 

One  typical  example  of  MED  usage  is  cold-potato  routing.  Assuming  MEDs 
are  not  used  (i.e.,  ignoring  step  3),  the  route-selection  procedure  above  (via  step  4) 
breaks  ties  based  on  closest  egress  point  (minimal  IGP  distance).  This  is  known 
as  hot-potato  routing.  Depending  on  the  destination  prefix,  a  neighboring  AS  may 
specify  alternate  preferences  for  ingress  points  using  the  MED  attribute,  e.g.,  to 
avoid  using  expensive  intra-domain  links.  Consider  two  inter-AS  connections:  one 
in  San  Erancisco,  one  in  New  York.  A  small  customer  network  may  have  high  costs 
sending  traffic  across  its  internal  links.  When  advertising  destinations  to  its  Internet 
provider,  the  customer  can  attach  appropriate  MED  values  to  the  destinations  so 
that  the  provider  chooses  the  egress  point  in  California  or  New  York  closest  to  the 
destination.  If  the  provider  instead  used  basic  hot-potato  routing,  the  closest  egress 
point  in  the  provider  network  would  be  chosen,  possibly  causing  the  customer  to 
handle  transcontinental  traffic. 
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In  Figure  1,  IGP  distances  are  listed  as  numbers  next  to  links;  MED  values  are 
listed  next  to  inter-AS  connections  in  parentheses.  Let  the  fixed  destination  be  AS 
0,  and  assume  that  all  paths  have  the  same  local-preference  value  assigned  at  AS 
3.  The  selection  functions  for  the  internal  routers  A  and  B  are  also  shown. 

Selection  functions  for  routers 
A  and  B: 

o-^(AC'10,  AL»20)  =  AD20 
a^^{AD20,  ABE20)  =  ABE20 
a^j^{ACW,ABE20)  =  ACIO 
a'^^{AC10,  AD20,  ABE20)  =  ACIO 


a%{BAD20,BE20)  =  BE20 
a%{BACW,BE20)  =  BACW 


Figure  1:  The  GSPP  MED-EVIL. 

This  instance,  called  MED-EVIL,  was  first  given  in  [8]  as  an  example  of  a  MED- 
induced  oscillation.  It  is  important  to  note  that  both  selection  functions  have  IRR 
violations  because  of  the  MED  values  set  by  AS  2;  thus,  the  paths  cannot  be  ranked 
and  this  configuration  cannot  be  represented  as  a  standard  SPP 

We  now  briefly  describe  why  fhis  GSPP  has  no  solufion.  Firsl  assume  fhaf  A 
and  B  have  nof  adverfised  roufes  fo  each  ofher;  fhen  fhey  will  choose  AD20  and 
BE20,  respectively,  because  of  minimal  IGP  disfances.  If  fhese  nodes  share  fhese 
choices,  B  will  still  choose  BE20  because,  even  fhough  BAD20  has  a  shorter 
IGP  pafh  lengfh,  ifs  MED  value  is  higher  fhan  BE20  and  bofh  pafhs  lead  fo  AS 
2.  Router  A,  upon  learning  of  ABE20,  will  no  longer  consider  AD20  because 
of  ifs  higher  MED  value  and  will  choose  ACIO  instead  (because  of  ifs  IGP  pafh 
lengfh  is  shorfer  fhan  ABE20,  fhe  ofher  viable  opfion).  When  A’s  new  choice  is 
broadcast  to  B,  router  B  will  choose  BACl^  because  of  its  shorter  IGP  distance 
(over  BE2{)),  withdrawing  BE2^.  However,  this  withdrawal  removes  the  path 
through  AS  2  with  lower  MED  value,  causing  A  to  choose  AD20  again,  withdraw¬ 
ing  AGIO.  Thus,  we  have  an  oscillation  similar  to  that  in  the  proof  above. 


AS  3 
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2.4  Generalized  Path- Vector  Policy  Systems 

The  Path- Vector  Policy  System  (PVPS)  framework  is  a  model  of  all  the  compo¬ 
nents  of  a  path- vector  protocol  [6] .  It  used  the  standard  SPP  as  a  semantic  domain 
for  protocol  expressiveness  and  included  path  rank  as  a  basic  component  of  the 
route-selection  procedure.  In  this  section,  we  expand  the  framework  to  include  ar¬ 
bitrary  selection  functions,  and  we  incorporate  GSPP  as  a  new  measure  of  protocol 
expressiveness.  The  expanded  framework  can  then  be  used  to  design  and  analyze 
non-IRR  path- vector  protocols. 

Definition  2.9.  A  generalized  path-vector  policy  system  ( GPVPS j  is  a  triple  com¬ 
prising:  PV,  the  path-vector  system  that  models  the  underlying  message -passing 
system  for  route  advertisements  and  signaling;  PL,  a  policy  language  used  to  con¬ 
figure  local-policy  inputs;  and  K,  a  global  constraint  on  network  instances  assumed 
to  be  true  for  executions  of  the  protocol  modeled  by  PV. 

The  P  V  is  described  by  several  components,  including: 

TZ:  the  set  of  allowable  path  descriptors,  the  data  structure  used  to  store  and  share 
routes; 

L*”,  L°“*:  constraints  on  import  and  export  policies,  which  are  transformations 
on  path  descriptors  used  to  change  the  attributes  of  paths  stored  in  routing 
tables; 

O:  a  constraint  on  originated  path  descriptors  (for  destinations  advertised  for  the 
first  time); 

L‘^:  constraints  on  nodes’  route- selection  functions;  and 

f*”,  the  policy-application  functions,  which  indicate  (1)  how  the  protocol  im¬ 

plements  the  application  of  nodes’  policies  to  path  descriptors  and  (2)  any 
built-in  transformations  performed  on  import  or  export. 

Remark  2.10.  By  fixing  some  consfrainfs,  a  GPVPS  can  be  resfricfed  fo  model  a 
linear  selecfion  function  and  a  sfandard  PVPS.  In  particular,  assume  fhere  is  some 
folally  ordered  sef  U  and  some  map  oj  :TZ  ^U,  and  lef 

lP{a)  ^  Vd,  a^[R)  =  min{P  |  uj[P)  <  cj(P')  VP'  E  P}. 

Definition  2.11.  An  instance  of  a  GPVPS  is  a  pair  I  =  {G,  F)  of  a  nefwork 
G  =  {y,  E)  and  a  configuration  function  F  fhaf  maps  each  u  E  V  fo  a  fuple 
comprising: 
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m™,  import  and  export  policies;  e.g.,  :  2^  — >  2^  is  a  function  / 

on  sets  of  path  descriptors  describing  the  transformation  performed  when  v 
imports  descriptors  from  u  such  that  L™(/)  holds  (analogously  for  and 

m'”'*®:  a  set  of  descriptors  originated  by  v  such  that  holds;  and 

a^:  the  route-selection  function  to  be  used  on  routing  tables  for  node  v  such  that 
holds. 

If  i?  is  a  set  of  path  descriptors  at  some  node  u,  then  those  path  descriptors  imported 
from  u  by  a  neighboring  node  v  can  be  represented  as  the  output  of  the  arc-policy 
function  for  the  signaling  edge  (n,  v).  This  function  combines  the  effect  of  export- 
and  import-policy  application  and  is  defined  as 

f(u,v){R)  =  i*”  {v,  U,  m'f-iu),  {u,  V,  R))  . 

A  path  assignment  p  :  1/  — >  2^  is  a  solution  to  the  instance  I  iff 

p{v)  =  ay  j  U  j  IJ  f(u,v){p{u)) 

y  \{u,v}&E 

Remark  2.12.  We  assume,  just  as  in  the  original  work  on  PVPSes  [6],  that  policy 
functions  are  separable,  i.e.,  if  p  is  a  policy  function  and  i?  is  a  set  of  path  descrip¬ 
tors,  then  p{R)  =  {p{r)  \  r  G  R}.  Separability  is  preserved  by  policy  application, 
thus  arc -policy  functions  are  separable: 

V  {u,  v)  E  E,  f(^u,v){R)  =  {f{u,v){r)  \  r  £  R}  . 

A  solution  therefore  must  consist  of  consistent,  best  paths  to  each  destination. 
We  make  the  following  definition  to  ease  the  notation  when  converting  paths  to 
path  descriptors  and  vice  versa. 

Definition  2.13.  Any  realizable  path  P  =  vviV2  ■  ■  ■  Vnd,  which  is  a  path  that  is 
contained  in  or  is  an  extension  of  a  path  realizable  at  the  next  hop  (r;i),  has 
a  corresponding  path  descriptor 

r{P)  =  o  f{v2,v^)  o  •  •  •  o  f{d,vy){r*), 

where  r*  £  TZ  is  the  path  descriptor  for  the  originated  destination  d.  If  P  is  not 
realizable  or  is  filtered,  r{P)  =  0. 

Conversely,  given  a  path  descriptor  r,  let  'P(r)  be  the  path  described  by  r. 
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Because  the  GPVPS  framework  is  general  enough  to  model  various  types  of 
inputs  and  constraints  on  those  inputs,  the  implementation  of  intended  protocol  be¬ 
havior  is  left  to  the  protocol  designer;  several  methods  may  achieve  the  same  types 
of  permitted  routing  configurations.  For  example,  the  general  notion  of  “routing- 
policy  expressiveness”  might  be  captured  by  either  allowing  broad  choices  of  se¬ 
lection  functions  for  individual  routers,  or  by  defining  a  simple  roufe-selecfion  pro¬ 
cedure  and  limifing  pafh-descripfor  fransformafions  via  imporf-  and  export-policy 
consfrainfs. 

Example  2.14.  To  model  BGP,  lef  fhe  sef  of  pafh  descriptors  be  fhe  sef  of  dafa 
records  used  in  BGP  updafe  messages  and  roufing  fables  (including  affribufes  such 
as  local  preference  and  MED).  Sef  fhe  policy-applicafion  functions  fo  hide  (or  zero- 
ouf)  all  privafe  aftribufes  on  eBGP  exporf  and  exfend  fhe  AS -pafh  enfry,  checking 
for  and  filfering  loops.  Then  sef  fhe  imporf  and  exporf  policy  consfrainfs  fo  limif 
affribufe  changes  as  described  in  fhe  BGP  specification  [12],  e.g.,  local  preference 
is  an  infeger  value  in  some  range.  Sef  fhe  origination  consfrainf  fo  check  for  fhe 
proper  form  of  pafh  descripfors  for  local  pafhs.  Finally,  sef  fhe  selecfion  funcfion 
consfrainf  so  fhaf  all  roufers  use  fhe  BGP  selection  procedure  described  earlier. 

The  original  framework  used  SPP  as  a  semanfic  domain  fo  measure  fhe  expres¬ 
siveness  of  a  PVPS,  i.e.,  fhe  fypes  of  roufing  configurations  fhaf  could  be  expressed 
using  a  PVPS  given  ifs  consfrainfs.  For  example,  a  single-desfination  nefwork  in- 
sfance  of  a  shorfesf-pafhs  PVPS  musf  be  consisfenf  wifh  an  SPP  in  which  each 
node’s  ordering  on  pafhs  mafches  fhe  pafh  lengfh.  Because  we  allow  arbifrary 
selecfion  funcfions  for  GPVPSes,  we  define  a  new  measure  of  expressiveness  in- 
corporafing  GSPP 

Definition  2.15.  Suppose  fhaf  1^  is  a  restriction  of  a  GPVPS  insfance  I  in  which 
fhe  only  pafh  descripfor  originafed  is  r  for  some  single  desfinafion  d.  Define  fhe 
GSPP  S{Ir)  to  have  the  set  of  permitted  paths  at  each  node  be  the  realizable  paths 
at  that  node,  i.e.,  V  =  =  v  ■  ■  ■  d  \  r{P)  /  0}.  (These  are  the  unfiltered 

paths.)  Let  the  selection  functions  be  the  same  as  in  the  GPVPS  instance.  Then 
S{Ir)  represents  the  routing  configuration  for  that  destination. 

The  expressive  power  of  a  GPVPS  P  F  is  the  set  of  all  allowable  routing  con¬ 
figurations  (all  allowable  instances),  i.e., 

A4(PV)  =  {5(/r)  I  Ir  is  a  restriction  of  a  legal  instance  I  of  PV}. 

Remark  2.16.  We  note  that  any  solution  vr  for  the  GSPP  S{Ir)  corresponds  to  a 
solution  p  for  the  restricted  GPVPS  instance  Ir  and  vice  versa.  The  proofs  of  these 
facts  are  mostly  algebraic  manipulation  and  exactly  mirror  the  analogous  proofs 
in  [6]  for  the  original  SPP  and  PVPS. 
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GSPP  can  also  be  used  as  a  better  measure  of  expressiveness  for  the  original 
PVPS;  all  GSPPs  in  this  case  will  have  linear  selection  functions  because  the  orig¬ 
inal  PVPSes  do  not  allow  IRR  violations. 

The  original  framework  defined  expressiveness  in  terms  of  equivalence  classes 
of  SPPs  because  only  the  relative  preference  ordering  of  paths  at  each  node,  not 
the  actual  integer  ranking  function  used  to  order  paths,  is  important  in  capturing 
a  routing  configuration.  Therefore,  any  SPP  S  belongs  to  an  equivalence  class  of 
SPPs  S{S)  in  which  all  SPPs  may  have  different  integer  rank  functions  but  have  the 
same  ordering  of  permitted  paths  at  each  node.  We  note  that  there  is  an  injection 
from  SPP  equivalence  classes  to  GSPPs;  there  is  a  canonical  GSPP  whose  selection 
function  orders  paths  in  the  same  way  as  each  of  the  SPPs  in  the  equivalence  class. 

Another  way  to  see  this  is  from  the  definition  of  linear  selection  functions  (Def¬ 
inition  2.3).  Given  a  linear  selection  function,  there  are  any  number  of  rank  maps 
io  that  preserve  the  choices  of  the  selection  function.  Every  one  of  these  rank  as¬ 
signments  corresponds  to  a  possible  SPP  instance  (because  nodes  have  functions 
assigning  ranks  to  paths),  but  these  SPPs  all  share  the  same  relative  preference  be¬ 
tween  paths  and  thus  belong  to  the  same  equivalence  class.  However,  there  is  only 
one  (canonical)  GSPP  for  a  set  of  selection  functions  assigned  to  nodes. 

2.5  GSPP  and  GPVPS  Convergence  Properties 

We  are  not  only  interested  in  whether  policies  interact  such  that  there  is  a  stable 
path  assignment,  i.e.,  whether  or  not  a  GSPP  has  a  solution,  but  also  in  how  path- 
vector  protocols,  following  the  three-step  hop-by-hop  process  described  above,  can 
reach  that  assignment.  In  the  next  section  we  will  provide  a  broad  sufficient  condi¬ 
tion  that  guarantees  robust  protocol  convergence  to  a  unique  solution;  the  condition 
can  be  used  as  a  global  constraint  for  GPVPSes.  To  derive  this  condition,  we  must 
investigate  protocol  behavior  in  addition  to  the  existence  of  solutions.  The  follow¬ 
ing  structure,  a  graph  constructed  from  a  GSPP  instance,  allows  us  to  do  this. 

To  simplify  our  discussion,  we  can  assume  that  routing  to  different  destinations 
are  computed  completely  independently;  therefore,  we  can  always  discuss  proto¬ 
col  convergence  with  respect  to  one  destination.  This  allows  us  to  use  restricted 
GPVPS  instances  and  GSPPs  to  describe  protocol  convergence  in  general.  If  a  par¬ 
ticular  convergence  property  holds  for  all  GSPP  instances  in  the  expressiveness  of 
a  GPVPS,  we  can  then  say  the  GPVPS  itself  has  that  convergence  property. 

Definition  2.17.  The  evaluation  digraph  of  a  GSPP  instance  S'  is  a  directed  graph 
T(S)  =  {Vr,  Eq-)  in  which  the  nodes  represent  protocol  selection  states,  and  the 
edges  represent  transitions  between  states.  A  selection  state  is  a  path  assignment 
^  ^  if  ®  £  ^T,  then  we  write  the  path  assignment  corresponding 
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to  this  node  as  tTo.  The  start  state  is  the  node  corresponding  to  the  empty  path 
assignment,  in  which  7r{vo)  =  (vq)  and,  for  v  ^  no,  7r(n)  =  e,  the  empty  path. 

The  directed  edge  (a,  (5)  is  present  in  Ej-  iff  for  aWv  ^  vq 


=  <°  U  {vvr„(n)} 

\{u,v}&E 

i.e. ,  given  that  nodes  select  the  paths  tTq  and  then  broadcast  these  selections  to  their 
neighbors  through  asynchronous  FIFO  links,  nodes  might  next  select  the  paths  vr^. 
Note  that  there  may  already  be  path  data  in  the  links  that  has  been  delayed  in 
transit,  so  that  7rc(n)  =  P  and  7r^(n)  =  P'  but,  for  a  neighbor  u,  Ea{u)  =  Q 
and  =  uP.  (Therefore,  states  may  not  be  consistent;  these  states  are  not 

acceptable  as  solutions.) 

We  can  follow  the  execution  of  a  path-vector  protocol  on  a  GSPP  instance  by 
its  trace,  which  corresponds  to  a  directed  path  in  the  evaluation  digraph  beginning 
at  the  start  state.  Traces  end  at  sink  states,  i.e.,  nodes  whose  only  outgoing  edges 
are  loop  edges.  Because  the  evaluation  digraph  is  finite,  if  all  traces  are  acyclic 
(ignoring  loop  edges),  then  all  protocol  runs  will  converge.  Conversely,  it  is  clear 
that  if  the  network  can  dynamically  oscillate  during  route  selection  then  there  is  a 
cycle  in  the  corresponding  evaluation  digraph;  each  of  the  paths  among  which  a 
node  oscillates  will  appear  in  at  least  one  of  the  states  in  the  corresponding  cycle. 

Proposition  2.18.  A  path  assignment  corresponds  to  a  sink  state  iff  it  is  a  solution. 

Proof.  A  solution  is  a  stable  set  of  consistent  routes.  Suppose  ttq,  is  a  solution;  then 
by  Definition  2.6,  for  all  v  vq  £  V,  By 

Definition  2.17,  this  is  equivalent  to  a  having  no  outgoing  edges  in  the  evaluation 
digraph  other  than  loop  edges,  meaning  that  a  is  a  sink  state.  □ 

Therefore,  we  can  define  profocol-convergence  properties  in  ferms  of  fhe  sfruc- 
fure  of  fhe  corresponding  evaluafion  digraph.  The  following  combinafions  of  fhe 
exisfence  of  solufions  and  fhe  abilify  of  profocols  fo  reach  fhose  solutions  are  of 
inferesf  fo  us. 

Definition  2.19.  The  following  are  convergence  properties  for  GSPP  and  GPVPS 
instances. 

Solvability  A  GSPP  is  solvable  if  there  exists  at  least  one  path  assignment  that  is 
a  solution;  i.e.,  the  evaluation  digraph  of  the  GSPP  has  at  least  one  sink  state. 
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Unique  Solvability  (Predictability)  A  routing  configuration  is  uniquely  solvable 
if  there  exists  exactly  one  GSPP  path  assignment  that  is  a  solution;  i.e.,  the 
evaluation  digraph  contains  exactly  one  sink  state. 

Safety  A  routing  configuration  is  safe  if  a  path- vector  protocol  is  able  to  converge 
to  a  solution;  i.e.,  all  traces  in  the  GSPP’s  evaluation  digraph  are  acyclic. 
The  existence  of  a  solution  does  not  determine  safety. 

Robustness  A  routing  configuration  is  robust  if  it  and  all  sub-instances  (resulting 
from  node  or  link  failures)  are  uniquely  solvable  and  safe;  i.e.,  all  traces  in 
the  GSPP  evaluation  digraph  are  acyclic  and  end  at  the  same  sink  state. 

Remark  2.20.  Note  that  the  definition  of  robustness,  while  requiring  all  sub¬ 
instances  to  be  predictable  and  safe,  requires  all  traces  only  in  the  original  GSPP’s 
evaluation  digraph  to  be  acyclic  and  end  at  the  same  sink.  This  is  because  sub¬ 
instances  have  evaluation  digraphs  that  are  subgraphs  of  the  original  instance’s 
evaluation  digraph  (with  some  paths  no  longer  possible  because  of  failures);  the 
property  of  acyclicity  holds  on  subgraphs. 

We  are  interested  in  robust  path-vector  protocols  because  these  avoid  nondeter¬ 
minism  and  divergence,  which  are  problems  that  are  difficult  for  network  operators 
to  understand  and  debug  when  they  occur  at  the  inter-domain  level. 

3  Main  Results:  Conditions  for  Protocol  Convergence 

Given  a  set  of  routing-policy  inputs,  we  can  study  the  corresponding  GSPP’s  eval¬ 
uation  digraph  to  see  how  they  affect  path- vector-protocol  execution.  However,  an 
evaluation  digraph  is  both  large  and  complex;  it  is  impractical  to  construct  it  as 
doing  so  requires  simulating  all  possible  update  sequences  in  the  GSPP  instance. 
Griffin,  Shepherd  and  Wilfong  [7]  showed  fhaf  a  smaller  sfrucfure,  called  a  dispute 
wheel,  can  be  consfrucfed  from  an  SPP  insfance  fhaf  is  nof  robusf.  Unforfunafely, 
fhe  original  definifion  of  fhe  sfrucfure  is  nof  compatible  wifh  nonlinear  selecfion 
functions. 

We  begin  fhis  secfion  by  infroducing  a  new  version  of  dispufe  wheels  and  prove 
fhaf  if  does  adequafely  capfure  oscillafions  in  generalized  SPPs.  From  fhaf  discus¬ 
sion,  we  are  fhen  able  fo  describe  oscillafions  in  ferms  of  an  underlying  order  on 
permiffed  pafhs  described  by  local-policy  configurations.  This  nofion  of  partially 
ordered  SPPs  firsf  appeared  in  [6];  however,  because  our  generalized  version  of  fhe 
problem  does  nof  have  a  nofion  of  rank,  we  musf  nonfrivially  change  fhe  compo- 
nenfs  of  fhis  order  fo  correcfly  describe  fhe  robusfness  condifion. 
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3.1  Generalized  Dispute  Wheels 


Definition  3.1.  A  generalized  dispute  wheel  (see  Figure  2)  contains  active  nodes 
Vo, ...  ,Vk  (with  all  subscripts  interpreted  modulo  A:  +  1)  such  that  Vi  has  a  spoke 
path  Qi  to  the  destination  d  and  Vi  and  Uj+i  are  connected  by  a  rim  segment  Ri+i 
such  that  either: 

1.353  {Qi,  Ri+iQi+i}  such  that  ct^.(5)  =  Ri+iQi+p,  or 
2.  3  5  ^  Ri+iQi+i  such  that 

(a)  a'^.  (5  U  {Qi})  /  Qi  and 

(b)  (5  U  {Qi,  Ri+iQi+i])  =  Qi,  or 

3.35^  Ri+\Qi+i  such  that 

(a)  (5  U  (Qil)  =  Qi  and 

(b)  U  {Qi,  i?i+iQi+i})  0  {Qi,  i?i+iQi+i}. 


''i-1 , 


z+i 


X  ; 


Figure  2:  Dispute  wheel. 


Remark  3.2.  Note  that  of  the  three  relationships  between  active  nodes  in  a  gener¬ 
alized  dispute  wheel,  only  condition  (1)  can  occur  for  a  linear  selection  function; 
conditions  (2)  and  (3)  imply  the  existence  of  an  IRR  violation.  Condition  (1)  is 
analogous  to  the  condition  on  rim  segments  found  in  the  original  definition  of  a 
dispute  wheel  for  standard  SPPs. 
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The  dispute  wheel  is  a  graph  constructed  from  a  GSPP  instance,  using  the  same 
vertices  and  edges  in  the  instance’s  network  graph.  However,  nodes  may  appear 
more  than  once  in  a  dispute  wheel,  e.g.,  in  more  than  one  spoke  path. 

Theorem  3.3.  If  the  evaluation  digraph  of  a  GSPP  instance  contains  a  cyclic  trace, 
i.e.,  if  a  GSPP  instance  is  not  safe,  then  it  contains  a  generalized  dispute  wheel. 

Proof  Let  C  be  a  cycle  in  the  evaluation  digraph  of  the  instance,  vq  a  node  which 
does  not  select  the  same  route  throughout  C,  and  Qq  one  of  the  paths  that  vq  selects 
in  C.  Without  loss  of  generality,  we  may  assume  that  u  is  the  last  (and  thus  only) 
node  on  Qq  that  does  not  select  the  same  route  throughout  C.  Viewing  Qo  as  one 
of  the  spokes  of  a  generalized  dispute  wheel,  we  now  attempt  to  construct  another 
such  spoke  and  a  rim  segment  joining  it  to  the  spoke  Qq. 

Let  vqPi  be  the  next  path  that  vq  selects  in  C,  and  let  xi  be  the  first  node  on 
Pi.  If  xi  oscillates  its  path  selection  in  C,  then  let  vi  be  the  last  cycling  node  on 
Pi,  let  Qi  =  vi  -  ■  ■  d  be  the  next  spoke,  and  Pi  =  vqxi  •  •  •  ni  be  the  rim  segment 
connecting  these  two  spokes.  (Both  Qi  and  Pi  are  subpaths  of  Pi.)  Because  xi 
oscillates  in  C,  it  must  broadcast  and  withdraw  Pi  during  the  oscillation,  and  one 
of  these  actions  causes  the  selection-state  transition;  thus  the  rim  segment  satisfies 
condition  (1)  in  Definition  3.1. 

If  xi  does  not  oscillate  in  C,  let  V0P2  be  the  path  that  vq  selects  in  C  after 
no  Pi  and  X2  the  first  node  on  P2.  If  X2  cycles  in  C,  we  may  proceed  as  above, 
otherwise  we  consider  the  path  V0P3  that  no  selects  in  C  after  V0P2,  etc.  Eventually, 
we  either  construct  another  spoke  connected  to  Qq  by  a  new  rim  segment  or  we 
progress  through  all  of  C  and  return  to  the  path  assignment  in  which  no  selects 
noPi.  If  the  latter  happens,  then  no  cycles  through  a  sequence  of  paths  in  C,  and 
each  of  these  paths  is  learned  from  a  neighbor  who  does  not  cycle  in  C.  All  of 
these  paths  are  thus  known  to  no  at  all  times,  therefore  all  of  the  changes  in  path 
assignment  to  vq  must  be  the  result  of  IRR  violations.  (This  is  because  a  change 
in  path  assignment  requires  that  no  know  of  different  routes  before  and  after  the 
change.  If  the  change  selects  a  route  that  was  already  known  but  not  chosen,  by 
Definition  2.2,  the  selection  function  for  vq  has  a  type-1  IRR  violation.) 

In  this  case,  assume  that  no’s  selection  of  Qq  is  the  result  of  =  Qo 

and  no’s  choice  of  noPi  is  the  result  of  cry^(Si)  =  vqPi,  with  QqjVqPi  S  (5  n 
Pi).  Because  PA  Pi  /  0,  there  is  some  route  P2  such  that  either  leai'ning  or 
withdrawing  noP2  causes  the  transition  from  P  to  Pi  and  Qo  to  noPi.  Let  x  be  the 
first  node  on  P2  and  ni  be  the  last  oscillating  node  on  P2.  (There  is  such  a  node 
because  P2  is  broadcast  and  withdrawn  in  the  oscillation;  otherwise  we  would 
not  have  this  oscillation.)  Then  we  can  let  Qi  =  vi  ■  ■  ■  d  be  the  next  spoke,  and 
Pi  =  vqx  •  •  •  ni  be  the  rim  segment  joining  them  such  that  either  condition  (2) — if 
P2  is  learned — or  condition  (3) — if  P2  is  withdrawn — is  satisfied. 
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Because  the  oscillation  cycle  is  finite,  we  can  repeat  this  process  until  we  reach 
a  selection  state  or  path  assignment  that  we  have  already  visited.  At  this  point,  a 
subset  of  the  spoke  and  rim  segments  will  form  a  generalized  dispute  wheel.  □ 

Corollary  3.4.  If  an  instance  of  GSPP  is  not  solvable,  then  it  contains  a  general¬ 
ized  dispute  wheel. 

Proof  An  unsolvable  GSPP  has  no  sink  state  in  its  evaluation  digraph;  therefore 
all  traces  must  contain  cycles,  and  any  of  these  cyclic  traces  produces  a  generalized 
dispute  wheel  by  Theorem  3.3.  □ 

Proposition  3.5.  If  an  instance  of  GSPP  has  multiple  solutions,  then  it  contains  a 
generalized  dispute  wheel. 

Proof.  We  follow  an  analogous  proof  method  in  [7].  Suppose  'Ki,'K2  are  two  so¬ 
lutions;  we  can  view  these  as  trees  in  the  network,  rooted  at  the  destination  vq'. 
Ti  =  IJ^gy  Then  let  H  =  {V,  E{Ti)  n  E{T2))  be  the  graph  induced  by  the 
intersection  of  the  trees  and  let  T  be  the  component  of  H  including  vq.  Note  that 
V  —  V{T)  is  nonempty — otherwise  Ti  =  T2. 

In  the  following  process,  assume  that  all  nodes  Ui  are  assigned  paths  in  both 
solutions.  Choose  an  edge  {ui,  r;!}  G  Ti  where  ui  0  V{T)  and  vi  G  V{T).  Then 
7ri(rii)  =  uiQi,  where  Qi  is  the  path  in  T  from  vi  to  d;  7ri(r;i)  =  7r2(ni)  =  Qi 
so  that  Qi  is  in  both  solutions  because  T  is  the  intersection  of  both  solutions. 
There  is  some  other  path  Pi  =  7r2(ui)  in  T2;  this  path  is  of  the  form  R2Q2  where 
i?2  =  ui  -  ■  ■U2  contained  in  T2  \  77  and  Q2  =  V2  -  ■  ■  d  contained  in  T.  Note 
that  'JT2{u2)  =  U2Q2,  so  we  can  repeat  this  process  by  examining  the  path  tti{u2). 
Continuing,  we  can  alternate  between  both  solutions  until  we  repeat  a  node  m. 

The  paths  7?*,  Qi  form  a  generalized  dispute  wheel.  This  is  because  for  each  i, 
there  must  exist  some  5  C  Vm  such  that  (T"°(5u{72j+i(5i+i,  WiQi})  =  Ri+iQi+i 
because  for  either  f  =  1  or  z  =  2,  7ri{ui)  =  7?j+iQj_|_i  given  the  construction 
above.  (If  not,  vTj  is  not  a  stable  solution,  because  the  path  UiQi  must  be  available 
given  that  Qi  is  in  the  intersection  of  both  solutions.)  This  satisfies  condition  (1)  in 
Definition  3.1.  □ 

The  contrapositive  of  the  above  three  assertions  forms  a  sufficient  condition  on 
GSPP  instances  that  guarantees  robust  protocol  convergence;  we  summarize  this 
as  the  following. 

Proposition  3.6.  If  a  GSPP  instance  has  no  generalized  dispute  wheel,  it  is  robust. 
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3.2  Partially  Ordered  GSPPs  and  Generalized  Dispute  Digraphs 

The  three  types  of  conditions  described  in  Definition  3.1  that  connect  dispute- 
wheel  spokes  by  rim  segments  can  be  used  to  define  relafions  befween  permiffed 
pafhs  in  a  GSPP.  These  relafions  can,  in  furn,  be  used  fo  define  a  graph  sfrucfure 
on  fhe  pafhs  in  a  GSPP  fhaf  makes  fhe  relationship  befween  pafhs  based  on  policy 
inferacfions  easy  fo  visualize.  The  underlying  compafibilify  of  local-policy  con¬ 
figurations  can  fhen  be  described  as  fhe  exisfence  of  a  consisfenf  partial  order  on 
permiffed  pafhs  using  fhese  relafions.  By  rephrasing  fhe  sufficienf  condifion  from 
Proposition  3.6  using  fhese  ferms,  we  can  better  undersfand  how  individual  pol¬ 
icy  inferacfions  (corresponding  fo  fhe  following  pafh  relafions)  consfifufe  a  global 
roufing  anomaly. 

Definition  3.7.  Define  fhe  following  four  relafions  on  permiffed  pafhs  in  a  GSPP 
insfance;  assume  fhaf  vq  is  fhe  fixed  desfinafion  node  and  fhaf  u,v  ^  V  we  ofher 
nefwork  nodes. 

Subpath  Pi  ©  P2  iff 

Pi  =  V  ■  ■  -  Vo,  P2  =  u  -  ■  -  Vo,  and  uPi  =  P2 

Linear  Selection  Pi  0  P2  iff 

Pi  =  V  ■  ■  ■  Vo,  P2  =  u  ■  ■  ■  Vo,  and  3  S  :  {{uPi,  P2}  U  S')  =  uPi 

Nonlinear  Selection  (first  type)  Pi  0i  P2  iff  Pi  =  v  ■  ■  ■  vo,  P2  =  u  -  ■  -vo,  and 

3S  ^  uPi  :  C  ({^2}  U  S)  /  P2  and  ({uPi,  P2}  U  S)  =  P2 

Nonlinear  Selection  (second  type)  Pi  ©2  P2  iff  Pi  =  v  ■  ■  ■  vq,  P2  =  u  -  ■  -vo,  and 
3S  ^  uPi  :  C(S)  =  P2  and  ({uPi}  U  S)  0  {uPi,  P2] 


We  now  define  fhe  following  graph  on  fhe  sef  of  permitted  pafhs  using  fhe 
above  relafions. 

Definition  3.8.  Given  a  GSPP  insfance  S,  ifs  generalized  dispute  digraph  is  fhe 
direcfed  graph  P(S)  =  {Vx>,Ex>)-  The  nodes  V®  =  P  are  fhe  permiffed  pafhs 
in  fhe  nefwork.  The  direcfed  edge  (Pi,  P2)  is  presenf  in  Ex>  iff  one  of  Pi  ©  P2, 
Pi  0  P2,  Pi  01  P2,  or  Pi  02  P2  holds. 
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Note  that  the  dispute  digraph  is  smaller  than  the  evaluation  digraph  as  each 
node  is  labeled  with  a  single  network  route  rather  than  a  set  of  network  routes;  it  is 
also  easy  to  build  given  the  definition  of  each  node’s  selection  function. 

Because  the  relations  correspond  to  transitions  in  the  evaluation  digraph  and 
connections  between  dispute-wheel  spokes,  we  can  prove  the  following. 

Theorem  3.9.  A  GSPP  instance  has  a  generalized  dispute  wheel  iff  it  has  a  cycle 
in  its  generalized  dispute  digraph. 

Proof.  First  assume  that  the  instance  has  a  generalized  dispute  wheel.  Its  rim  gives 
a  cycle  in  the  generalized  dispute  digraph  as  follows,  because  the  pair  of  paths 
from  adjacent  rim  nodes  to  the  destination  each  belong  to  one  of  the  four  rela¬ 
tions  in  Definition  3.7.  Begin  with  any  active  node  Vi  on  the  rim;  let  ri  be  the 
next  node  on  the  rim  segment  Ri.  From  the  construction  of  the  dispute  wheel, 
t'lQi  =  rin,  •  •  •  d  is  an  extension  of  Qi,  so  Qi  ©  rQp,  this  relation  holds  for 
further  extensions  along  the  rim,  such  that  (r*  •  •  •  riQi)  0  (rj+irj  •  •  •  riQi).  Let 
R*  be  the  rim  segment  up  to,  but  not  including,  Vi-p,  using  these  relations,  we 
see  there  is  a  path  from  Qi  to  R*  Qi  in  the  dispute  digraph  for  each  active  node 
Vi  in  the  dispute  wheel.  Call  these  paths  Di.  Then,  for  every  RiQi  and  Qi-i, 
one  of  the  three  conditions  in  Definition  3.1  holds.  In  the  case  of  condition  (1), 
3  5  :  o-jf._j(5  U  {RiQi,  Qi-i})  =  thus  corresponding  to  the 

edge  {R* Qi,  Qi-i)  connecting  Di  and  Di-i.  In  the  case  of  condition  (2),  learning 
RiQi  at  Vi-i  forces  another  route  to  be  selected  over  Qi-i',  thus  R*Qi  ©2  Qi-1, 
also  corresponding  to  the  edge  {R*Qi,  Qi-i)  connecting  Di  and  i3j_i.  Finally,  in 
the  case  of  condition  (3),  withdrawing  some  route  at  Vi-i  forces  Qi-i  to  be  cho¬ 
sen;  thus  R*Qi  ©1  Qi-i,  corresponding  to  the  same  edge  connecting  Di  and 
Therefore  the  dispute-digraph  edges  corresponding  to  pairwise  relations  between 
paths  starting  at  adjacent  rim  nodes  form  a  cycle. 

In  the  other  direction,  assume  that  we  have  a  cycle  in  the  dispute  digraph. 
Consider  any  edge  (Pi,  P2)  and  examine  the  relation  between  Pi  and  P2.  If  Pi  © 
P2,  then  let  the  first  node  of  Pi  be  a  rim  node  and  connect  it  to  the  first  node  of 
P2  as  an  adjacent  rim  node  (counterclockwise,  referencing  Figure  2.).  If  Pi  0  P2, 
Pi  01  P2,  or  Pi  02  P2,  then  let  P2  be  a  spoke  Qi  and  connect  the  first  node  of  P2 
to  the  first  node  of  Pi  on  the  rim  segment  Pj+i;  the  subpath  of  Pi  from  the  first 
node  to  the  last  oscillating  node  will  be  the  rim  segment  Pj+i  and  the  remainder 
of  Pi  will  be  the  next  spoke  Qi+i.  The  resulting  structure  will  obey  one  of  the 
three  conditions  in  Definition  3.1  for  rim  segments  connecting  spokes  and  will 
have  subpaths  along  individual  rim  segments  (moving  clockwise);  therefore,  this 
structure  is  the  dispute  wheel  corresponding  to  the  dispute-digraph  cycle.  □ 

This  immediately  leads  to  the  following  corollary,  which  provides  an  equiv- 
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alent  sufficient  condition  to  Proposition  3.6  using  the  transitive  closure  of  path 
relations  (local  conditions)  instead  of  dispute-wheel  freeness  (a  global  condition). 

Corollary  3.10.  Given  a  GSPP  instance,  if  there  is  a  cycle  in  its  evaluation  di¬ 
graph,  then  the  corresponding  relation  O  =  (©  U  0  U  Oi  U  02)*  on  permitted 
paths  is  not  a  partial  order. 

Proof.  If  Pi  Q)  P2,  then  there  is  a  path  in  the  dispute  digraph  from  Pi  to  P2  (as  in 
the  proof  of  Theorem  3.9).  If  the  relation  Q  is  not  a  partial  order,  there  are  two 
paths  Pi  7^  P2  such  that  Pi  Q  P2  and  P2  O  A ;  the  two  corresponding  paths  form  a 
dispute-digraph  cycle.  Analogously,  if  there  is  a  cycle  in  the  dispute  digraph,  there 
are  paths  Pi  /  P2  corresponding  to  nodes  in  this  cycle  such  that  Pi  Q  P2  and 
P2Q)  Pi,  thus  O  cannot  be  a  partial  order.  The  result  then  follows  directly  from 
Theorems  3.3  and  3.9.  □ 

Remark  3.11.  The  original  set  of  relations  defined  in  [6]  for  SPP  partial  ordering 
could  assume  linear  selection  functions;  thus  both  types  of  the  “nonlinear  selec¬ 
tion”  relation  were  not  used.  However,  the  “linear  selection”  relation  was  also 
different,  defined  as  follows:  assuming  that  u;  is  a  ranking  function.  Pi  0  P2  iff 
a;(Pi)  <  u}{P2).  In  this  version  of  the  definition,  both  paths  begin  at  the  same 
node,  and  the  extension  of  Pi  to  u  in  Definition  3.7  was  captured  in  the  transitive 
closure  of  0  with  the  subpath  relation  ©.  If  we  defined  a  similar  selection  relation, 
i.e.,  Pi  0  P2  iff  there  exists  some  S  such  that  (t({Pi,  P2}  U  5)  =  Pi,  then  any 
IRR  violation  would  automatically  introduce  a  cycle  in  the  dispute  digraph  (this 
fact  follows  directly  from  Definition  2.2).  Because  not  all  such  IRR  violations 
cause  protocol  oscillations  (given  other  nodes’  policies),  subsuming  one  subpath 
relation  into  the  three  selection  relations  eliminates  these  spurious  cycles  from  dis¬ 
pute  digraphs.  Consequently,  the  example  dispute  cycles  in  the  next  subsection 
will  appear  different  than  in  [6,7]. 

Using  the  generalized  dispute  digraph,  one  can  diagnose  the  cause  of  oscilla¬ 
tions:  cycles  that  involve  nonlinear- selection  edges  are  the  result  of  IRR  violations, 
and  cycles  only  involving  linear-selection  and  subpath  edges  are  manifestations  of 
basic  policy  disputes  not  based  on  IRR  violations. 

3.3  Example  GSPPs  and  Dispute  Digraphs 

In  the  following  diagrams  of  generalized  dispute  digraphs,  we  will  use  the  follow¬ 
ing  convention  for  edges:  solid  lines  correspond  to  subpath  relations,  dashed  lines 
correspond  to  linear-selection  relations,  and  dotted-and-dashed  lines  correspond  to 
nonlinear- selection  relations.  Of  these,  those  with  solid  arrowheads  are  of  the  first 
type  while  those  with  white  arrowheads  are  of  the  second  type. 
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Example  3.12.  We  begin  with  the  generalized  dispute  digraph  for  MED-EVIL,  the 
GSPP  from  Example  2.8.  To  simplify  the  diagram,  we  have  condensed  ASes  1,  2, 
and  0  into  a  single  AS  0  connected  to  routers  C,  D,  and  E\  we  can  write  analogous 
selection  functions  that  maintain  the  policies  and  MED-induced  oscillation  in  the 
original  MED-EVIL.  The  graph  is  shown  in  Eigure  3. 
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Eigure  3:  Generalized  dispute  digraph  for  MED-EVIL. 

This  digraph  has  two  cycles,  AGO  —  BEO  and  ADO  —  BED',  as  expected,  both 
of  these  involve  nonlinear  selection  edges  and  the  paths  that  cause  IRR  violations. 
The  policies  of  MED-EVIL  create  no  oscillation  when  MEDs  are  ignored:  note 
that  the  linear-selection  edges  do  not  form  any  cycles.  Involving  MEDs  creates 
relations  between  paths  that  are  not  consistent  with  a  partial  order.  To  achieve  a 
partial  order,  we  can  attempt  to  change  local  policies  to  change  the  relations,  i.e., 
break  the  cycle;  e.g.,  we  can  force  node  A  to  always  choose  the  path  ACO. 

Example  3.13.  We  now  revisit  two  canonical  policy-induced  oscillations  repre¬ 
sented  by  SPP  instances  first  given  by  [7].  The  instance  DISAGREE  is  shown  in 
Eigure  4;  it  contains  two  stable  solutions  but  does  not  predictably  converge  to  ei¬ 
ther  one,  thus  its  dispute  digraph  contains  a  cycle.  The  instance  BAD  GADGET  is 
shown  in  Eigure  5;  it  has  no  solution,  so  its  dispute  digraph  also  contains  a  cycle. 
Because  these  instances  have  linear  selection  functions,  the  route-selection  func¬ 
tions  are  shown  as  a  linear  order  on  permitted  paths  next  to  each  node  such  that  the 
most  preferred  path  is  listed  on  top. 
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(a)  (b) 

Figure  4:  (a)  The  SPP  instance  DISAGREE  and  (b)  its  corresponding  generalized 
dispute  digraph. 


Figure  5 :  (a)  The  SPP  instance  BAD  GADGET  and  (b)  its  corresponding  generalized 
dispute  digraph. 


4  Applications  to  Protocol  Design 

In  this  section  we  examine  some  strategies  for  constraining  policies  to  guarantee 
robust  protocol  convergence.  Although  dispute  wheels  and  dispute  digraphs  are 
useful  tools  for  studying  policy  interactions,  using  them  can  be  impractical  for  real 
network  configurations.  The  dispute  digraph  has  size  proportional  to  the  number 
of  loopless  paths  in  a  network;  checking  for  dispute  wheels  is  at  least  as  hard, 
because  there  is  no  known  way  to  directly  produce  a  dispute  wheel  without  an 
instance’s  dispute  digraph  or  evaluation  digraph.  Furthermore,  it  is  almost  impos¬ 
sible  to  obtain  Internet-wide  policy  information  to  generate  these  structures,  and 
the  structures  will  be  different  every  time  nodes  make  policy  changes.  Ideally, 
we  want  constraints  on  the  protocol  specification  or  policy-configuration  language 
that  applies  to  a  broad  set  of  networks  and  routing  configurations — we  would  like 
to  use  the  sufficient  condition  from  the  previous  section  while  allowing  for  as  much 
policy  expressiveness  as  possible. 
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Previous  work  [5,  6,  13]  has  given  concrete  local  constraints  on  policies  that 
guarantee  robustness.  Unfortunately,  generalizing  these  types  of  constraints  is  hard 
because  our  new  model  allows  for  nonlinear  selection  functions,  which  removes 
any  notion  of  path-rank  values,  and  because  our  model  broadens  the  notion  of  the 
protocol’s  route-selection  procedure  arbitrarily. 

Some  obvious,  draconian  constraints,  e.g.,  preventing  the  advertisement  of  any 
route  that  causes  an  IRR  violation,  can  be  trivially  shown  to  prevent  routing  anoma¬ 
lies,  but  these  are  very  strict  and  harshly  limit  expressive  power.  Below  we  review  a 
specific  proposal  to  review  MED-induced  oscillations  in  BGP,  and  we  use  our  tools 
to  suggest  an  improvement.  In  the  following  subsections,  we  discuss  two  other  con¬ 
jectured  solutions  and,  using  the  results  from  the  PVPS  and  GPVPS  frameworks, 
prove  them  to  be  true. 

4.1  Multiple-Path  Broadcast 

Basu  etal.  [1]  and  Musunuri  and  Cobb  [11]  proved  that  a  modification  to  BGP’s  up¬ 
date  messages  will  prevent  MED-induced  oscillations.  They  suggested  that  nodes 
broadcast  not  only  best  routes,  but  any  route  that  remains  after  step  3  in  the  BGP 
route-selection  process  (see  Example  2.8),  i.e.,  all  routes  with  minimal  MED  val¬ 
ues,  possibly  one  for  each  AS,  are  broadcast,  not  only  the  one  with  minimal  IGP 
distance  to  the  egress  point.  This  prevents  routes  that  cause  IRR  violations  from 
being  broadcast  and  withdrawn  repeatedly.  In  the  case  of  MED-EVIL  in  Exam¬ 
ple  2.8,^  node  B  would  then  always  broadcast  the  route  BE20,  even  though  it 
would  never  select  it.  Because  the  route  is  never  chosen  elsewhere  due  to  its  high 
MED  value,  this  introduces  no  consistency  problems.  However,  it  (1)  allows  other 
nodes  to  make  the  correct  choice  of  routes  with  respect  to  MED  values  and  (2) 
stops  the  oscillation  by  making  that  choice  stable. 

We  can  see  the  effect  of  such  a  change  by  examining  the  cyclic  traces  in  the 
evaluation  digraph.  The  MED-induced  cycle  of  MED-EVIL  is  shown  in  Eigure  6. 
The  nodes  show  the  selections  of  nodes  A  and  B,  and  the  labels  on  arrows  show 
the  causes  of  transitions  (routes  being  advertised  or  withdrawn).  Note  the  IRR  vio¬ 
lation  is  clear  in  the  transition  between  the  first  and  second  states;  node  A  switches 
from  ADO  to  ACO  by  learning  a  different  route,  ABEO.  With  multiple-path  broad¬ 
cast,  the  withdrawal  of  ABEO  never  takes  place;  therefore  the  state  (ACO,  BACO) 
becomes  a  sink  state  and  a  stable  assignment. 

This  effect  easily  generalizes  to  all  GSPP  instances  involving  MEDs:  any 
MED-induced  oscillation  corresponds  to  an  evaluation-digraph  cycle  of  the  above 
form,  and  preventing  a  route  withdrawal  by  broadcasting  additional  routes  will 

^As  in  Example  3.12,  we  simplify  the  instance  by  condensing  AS  1,  AS  2,  and  AS  0  into  a  single 
AS  0  and  modifying  the  selection  functions  accordingly. 
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Figure  6:  Cycle  in  the  evaluation  digraph  of  MED-EVIL. 


break  the  cycle  by  preventing  one  (or  more)  of  the  cycle’s  transitions.  In  addition, 
because  more  routes  are  always  broadcast,  nodes  will  not  choose  higher-MED- 
valued  routes  when  lower-MED-valued  routes  are  available,  thus  preserving  the 
intended  behavior  of  the  MED  attribute.  More  generally,  if  routes  causing  IRR  vi¬ 
olations  are  always  broadcast,  the  resulting  route-selection  functions  with  restricted 
domain  have  no  IRR  violations. 

Multiple-path  broadcast  can  increase  the  size  of  routing  tables  and  update  mes¬ 
sages.  However,  we  propose  that  IRR  violations  can  be  detected  dynamically,  pre¬ 
cisely  when  a  newly  learned  route  causes  a  switch  in  selection  without  selecting  the 
new  route.  Requesting  that  the  new  route  always  be  broadcast  will  prevent  a  future 
oscillation  due  to  withdrawal  of  that  route  without  any  route  inconsistencies.  Main¬ 
taining  one  extra  route  as  needed  is  more  storage-efficient  than  the  multiple-path 
broadcast  proposed  by  [1,11].  Although  this  solution  requires  further  modification 
to  BGP,  dynamic  detection  of  IRR  violations  is  possible  by  examining  protocol- 
execution  traces.  In  practice,  whenever  a  BGP  update  message  is  received,  the 
route  selection  before  and  after  the  update  message  can  be  compared.  If  the  new 
selection  is  neither  the  old  selection  or  the  newly  learned  route,  the  route  points  to 
an  IRR  violation  (this  is  clear  from  Definition  2.2.  Requesting  this  IRR-violating 
route  to  be  broadcast  as  long  as  it  is  available  prevents  any  induced  oscillations 
because  the  route  essentially  becomes  fixed,  breaking  fhe  cycle  of  wifhdrawals  and 
adverfisemenfs  in  fhe  evaluafion  digraph.  Eormally,  we  have  fhe  following. 

Proposition  4.1.  An  oscillation  due  to  an  IRR  violation  can  be  dynamically  de¬ 
tected  and  stopped  by  requesting  one  additional  route  to  be  broadcast  permanently. 

Proof.  Given  a  cycle  in  fhe  evaluafion  digraph  involving  an  IRR  violation,  fhere 
are  fransifions  in  fhis  cycle  involving  an  adverfisemenf  or  wifhdrawal  of  a  roufe 
fhaf  is  never  selecfed.  This  roufe  can  be  defecfed  by  comparing  pafh  assignmenfs 
in  fhe  sfafes  adjacenf  fo  fhese  fransifions.  If  fhe  wifhdrawal  fransifion  is  prevenfed 
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by  forcing  the  route  to  be  advertised  as  long  as  it  is  available,  even  if  it  is  not 
chosen,  the  withdrawal  transition  cannot  take  place  and  the  cycle  is  broken.  □ 

This  procedure  breaks  cycles  in  a  “snapshot  of  time,”  i.e.,  for  a  static  routing 
configuration  that  induces  a  protocol  oscillation.  If  changes  occur  and  routes  are 
introduced  or  withdrawn  for  legitimate  causes,  the  resulting  GSPP  instance  will 
have  a  different  evaluation  digraph;  however,  the  relevant  IRR-violating  routes  can 
be  detected  for  this  new  configuration  in  the  same  way.  If  the  IRR-violating  route  is 
no  longer  available,  the  broadcasting  node  can  send  the  appropriate  withdrawal — 
this  still  allows  the  receiving  node  to  detect  new  IRR  violations  involving  other 
routes.  Furthermore,  if  any  IRR-violating  selections  are  superseded  by  learning 
new  routes  that  are  always  more  preferred  or  by  other  IRR-violating  routes,  the 
original  routes  are  not  needed  and  the  broadcast  can  be  stopped. 

4.2  Compare  All  MEDs 

Some  routers  have  an  option  to  change  the  route-selection  procedure  involving 
MEDs:  In  step  3  of  the  BGP  procedure  described  in  Example  2.8,  instead  of  elimi¬ 
nating  multiple  paths  to  the  same  AS  by  choosing  the  one  with  lowest  MED  value, 
MED  values  are  compared  across  all  paths  so  that,  regardless  of  AS  next-hop,  only 
paths  with  the  lowest  MED  values  are  retained  for  possible  selection. 

This  option  essentially  changes  the  route-selection  procedure  so  that  it  is  linear: 
for  each  path,  the  preference  of  that  path  depends,  in  order,  on  its  local  preference, 
then  path  length,  then  MED  value,  and  finally  IGP  distance.  Therefore,  IRR  vio¬ 
lations  are  no  longer  possible,  and  previous  convergence  constraints  apply.  In  fact, 
because  local-preference,  AS -path  length,  and  MED  values  do  not  change  during 
intra-domain  BGP  (iBGP)  sessions,  and  because  IGP  distances  increase  as  paths 
are  extended,  the  absolute  rank  value  associated  with  paths  increases  on  extension. 
This  obeys  the  strict-monotonicity  constraints  of  [6, 13],  so  MED-induced  oscilla¬ 
tions  cannot  occur.  (Of  course,  more  general  policy-induced  oscillations  due  to, 
e.g.,  local -preference  settings,  can  still  occur.) 

Eormally,  comparing  all  MEDs  changes  the  route-selection  procedure  such 
that  selection  functions  are  linear,  compatible  with  the  rank  map  io{l,  P,m,d)  = 
{—I,  \  P\,m,d),  lexically  ordered,  where  I  is  the  local  preference,  P  is  the  AS  path 
(path  vector),  m  is  the  MED  value,  and  d  is  the  IGP  distance. 

4.3  AS-Distinct  Local-Preference  Settings 

McPherson  et  al.  in  [10]  suggest  a  workaround  for  MED-induced  oscillations  that 
prevents  BGP  from  having  a  conflict  when  it  reaches  the  MED  step.  If  only  routes 
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from  one  AS  remain  when  MEDs  are  considered,  then  all  routes  have  their  MED 
values  compared  and,  similar  to  above,  IRR  violations  are  not  possible.  One  simple 
way  to  do  this  is  to  assign  local-preference  values  such  that  no  two  routes  from 
different  ASes  have  the  same  value;  then  the  first  step  of  the  BGP  selection  process 
will  automatically  eliminate  all  routes  except  those  from  a  single  AS.  (One  can  also 
assign  distinct  local-preference  values  to  equidistant  ASes;  then  the  first  two  steps 
eliminate  all  routes  but  those  from  one  AS.) 

This  route-selection  procedure  is  consistent  with  linear  selection  functions  be¬ 
cause,  just  as  above,  the  rank  of  a  route  independently  depends  on  four  criteria  in 
order.  Once  the  MED  value  is  considered,  all  remaining  routes  have  the  same  local 
preference,  path  length,  next-hop  AS,  and  MED  value,  again  leaving  the  strictly 
monotonic  IGP  distance  to  be  used  to  break  ties.  Therefore,  this  modification  to 
BGP  prevents  MED-induced  anomalies. 

5  Conclusions  and  Future  Work 

In  this  paper,  we  have  fully  extended  the  notion  of  convergence  conditions  on  SPPs 
to  the  generalized  version  of  the  problem  allowing  arbitrary  route-selection  proce¬ 
dures  instead  of  those  based  on  some  notion  of  a  linear  path  rank.  Doing  so  allows 
us  to  fully  understand  the  causes  of  policy-induced  routing  anomalies  from  the  per¬ 
spective  of  an  underlying  mathematical  consistency  between  nodes’  policies.  The 
generalized  version  is  able  to  capture  the  behavior  of  protocols,  e.g.,  BGP  with 
MEDs,  that  do  not  satisfy  independent  route  ranking  (IRR).  In  examining  these 
generalized  policy  interactions,  we  rigorously  defined  profocol-convergence  prop¬ 
erties  and  several  useful  graph  structures  that  illustrate  protocol  behavior.  We  pro¬ 
vided  two  equivalent  sufficient  conditions  for  robust  convergence,  both  of  which 
involve  structures  that  are  significantly  smaller  than  examining  the  execution  states 
of  the  protocol  directly.  We  also  discussed  applications  of  these  results  to  proto¬ 
col  design,  including  some  simple  (but  strict)  local-policy  constraints  and  rigorous 
examinations  of  proposed  solutions  to  MED-induced  oscillations. 

There  are  two  obvious  directions  for  future  work  left  open  in  our  paper.  The 
first  is  the  development  of  analogous  constraints  to  monotonicity  for  arbitrary 
selection  functions.  The  original  condition  highly  depends  on  path-rank  values, 
which  are  not  available  in  the  generalized  version,  but  the  constraint  provided  an 
equivalent  local  condition  to  dispute-wheel  freeness.  We  have  only  been  able  to 
develop  very  simple  local-policy  constraints,  and  these  limit  expressiveness  quite 
a  bit.  Because  route-selection  procedures  are  so  broad,  we  expect  it  to  be  difficult 
to  generalize  previous  notions  of  constraints.  One  method  that  may  prove  fruitful 
is  examining  a  restricted  set  of  route-selection  procedures;  e.g.,  the  application  of 
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PVPSes  to  class-based  systems  [9]  produced  a  set  of  robust  protocols  with  a  vary¬ 
ing  but  concrete  balance  of  local  and  global  constraints.  Similar  work  could  be 
done  while  still  allowing  IRR  violations. 

The  second  is  a  further  broadening  of  the  model  to  capture  the  static  semantics 
of  policy  interactions  when  multiple-path  broadcast  is  used;  more  generally,  the 
results  can  be  extended  to  SPPs  with  set-valued  arbitrary  selection  and  broadcast 
functions.  The  notion  of  a  broadcast  function  is  similar  to  a  selection  function:  it 
returns  a  subset  of  paths,  as  dictated  by  the  protocol  specification,  that  are  adver¬ 
tised  to  neighbors,  given  nodes’  routing  tables  as  inputs.  (This  allows  more  lati¬ 
tude  in  modeling  step  3  of  path- vector-protocol  behavior  discussed  in  Section  2. 1 .) 
Separating  broadcast  functions  from  export  policies  allows  more  freedom  in  im¬ 
plementing  constraints  at  different  parts  of  the  route-advertisement  process  and 
keeps  the  separability  property  for  policy  functions.  Unfortunately,  it  seems  that 
the  problem  statement  for  arbitrary,  set-valued  selection  and  broadcast  functions 
is  difficult;  in  fact,  the  problem  of  finding  a  stable,  consistent  solution  for  such 
a  routing  configuration  may  not  be  in  NP  because  there  may  be  a  set  of  oscillat¬ 
ing  routing  tables  for  each  node  that,  given  particular  broadcast  functions,  do  give 
at  least  one  stable  solution.  This  extension  to  the  model  could  help  design  and 
evaluate  protocols  or  protocol  modifications  such  as  the  ones  reviewed  briefly  in 
Section  4.1  without  having  to  study  large  evaluation  digraphs. 
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